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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 



- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 



3) \Z\ Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-31 is/are pending in the application. 

4a) Of the above claim(s) 2,3 and 30 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1A-29 and 31 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 1 1 9 

1 2)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Status 



1 )K Responsive to communication(s) filed on 05 January 2004 . 
2a)^ This action is FINAL. 2b)D This action is non-final. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) Q Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 



4) 0 Interview Summary (PTO-413) 



5) □ Notice of Informal Patent Application (PTO-1 52) 

6) £3 Other: Detailed Office Action . 
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DETAILED ACTION 



Response to Amendment 
1. In Amendment "D" 5 filed January 5, 2004 (see paper no. 20), claims 1, 4-29 and 31 are 
presented for reconsideration. 



Response to Arguments 
2. Applicant's arguments filed January 5, 2004 have been fully considered but they are not 
persuasive. 



Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1, 4-29 and 31 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Jagielski, "An Application of Neural Networks to Emulation of Aesthetic Judgments" [hereafter 
Jagielski] in view of Yasuda et al., "Application of Neural Network to Aesthetic Design of 
Bridges" [hereafter Yasuda]. 

As per claim 1, Jagielski teaches of a computer-implemented method comprising: (A) 
inputting a training set including a plurality of images and a corresponding plurality of aesthetic 
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scores for the images (see Abstract; and page 336, column 1: inputting images and corresponding 
aesthetic values); (B) training a classifier to provide aesthetic scores based on the training set 
(see page 336, columns 1 and 2: training the classifier); and (C) generating an aesthetic score for 
the image based on the classifier (see page 336, columns 1 and 2: training the classifier and 
generating a score). Although Jagielski clearly teaches of using back-propagation in order to 
fine-tune the disclosed neural network for several test cases (see page 336, column 2, paragraph 
four through page 337, column one, paragraph one; and Tab 2 on page 337), the art fails to 
explicitly disclose the generation of a recommendation to improve the aesthetic score. 

However, Yasuda teaches of generating a recommendation to improve ah aesthetic score 
for the image by evaluating the end results and compiling a recommendation that can be applied 
to a design to improve the project (see sections 5-6 on pages 540-541, especially section 6 on 
page 541). 

It would have been obvious to one of ordinary skill in the art, at the time of the 
applicant's invention, to combine Jagielski 's method of generating an aesthetic score for input 
images based on a trained neural network, with Yasuda' s method of generating a 
recommendation based upon results from the network, in order to provide a method for 
presenting to the user and/or designer, a recommendation as to some of the aesthetic options, 
which when applied to an image/project, will provide greater aesthetic value. Merging the two 
methods also provides the user and/or designer the flexibility of either adding or deleting 
determined options to better refine an image/project. 

As per claim 4, Jagielski teaches of generating aesthetic scores (see claim one above), 
but fails to explicitly disclose generating recommendations by employing a gradient ascent. 
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However, Yasuda teaches of generating a recommendation that comprises the use of a 
gradient ascent during the system configuration by the neural network (see section 4.1 on page 
537: quantifying the multiple inputs into the neural network in order to facilitate rule generation 
for the resultant recommendation). 

It would have been obvious to one of ordinary skill in the art, at the time of the 
applicant's invention, to combine Jagielski's method of generating aesthetic scores with 
Yasuda' s method of employing a gradient ascent in order to provide a method for quantifying 
objective and subjective (e.g., qualitative and quantitative) input values, and generating a 
quantitative value in order to provide a numerical basis for the neural network to operate. 

As per claim 5, Jagielski teaches of generating aesthetic scores (see claim one above), 
but fails to explicitly disclose generating recommendations via local searches. 

However, Yasuda teaches of generating a recommendation comprising of performing a 
local search (see section 3 on page 535; and tables 1-3 on page 536: searching and employing 
aesthetic parameters along with local images). 

It would have been obvious to one of ordinary skill in the art, at the time of the 
applicant's invention, to combine Jagielski's method of generating aesthetic scores with 
Yasuda' s method of performing local searches in order to provide a more efficient [via both cost 
and resource usage] method of gathering information relative to a project, and forming a basis 
for generating the resulting recommendation. 

As per claim 6, Jagielski teaches of a method wherein training a classifier comprises 
training one of a Bayesian classifier, a Support Vector Machine (S VM) classifier, a neural net 
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classifier, and a decision tree classifier (see Abstract; and page 336, column 1 : employing a 
neural network for training purposes). 

As per claim 7, Jagielski teaches of a method wherein training a classifier comprises 
utilizing feature selection to correlate at least one image feature of the images with their 
corresponding aesthetic scores (see page 335, column 1: using aesthetic descriptors and 
preferences). 

As per claim 8, Jagielski teaches of a method wherein utilizing feature selection to 
correlate at least one image feature comprises utilizing feature selection to correlate at least one 
image feature selected from the group essentially consisting of: color presence, color 
distribution, geometrical quantities of segmented image parts, coefficients of image 
transformations, and higher-level image representations (see page 335, column 1: using aesthetic 
descriptors and preferences). 

As per claim 9, Jagielski teaches of a computer-implemented method comprising: (A) 
inputting an image (page 336, column 1); and (B) generating an aesthetic score for the image by 
utilizing a classifier previously trained on a training set including a plurality of images and a 
corresponding plurality of aesthetic scores for the images (see page 336, columns 1 and 2). 

Although Jagielski clearly teaches of using back-propagation in order to fine-tune the 
disclosed neural network for several test cases (see page 336, column 2, paragraph four through 
page 337, column one, paragraph one; and Tab 2 on page 337), the reference fails to explicitly 
disclose the generation of a recommendation to improve the aesthetic score. 

However, Yasuda teaches of generating a recommendation to improve an aesthetic score 
for the image by evaluating the end results and compiling a recommendation that can be applied 
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to a design to improve the project (see sections 5-6 on pages 540-541, especially section 6 on 
page 541). 

It would have been obvious to one of ordinary skill in the art, at the time of the 
applicant's invention, to combine Jagielski's method of generating an aesthetic score for input 
images based on a trained neural network, with Yasuda's method of generating a 
recommendation based upon results from the network, in order to provide a method for 
presenting to the user and/or designer, a recommendation as to some of the aesthetic options, 
which when applied to an image/project, will provide greater aesthetic value, while also 
presenting to the user and/or designer the flexibility to either add or delete aforementioned 
options. 

As per claim 10, Jagielski teaches of a method wherein generating an aesthetic score 
comprises generating an aesthetic score based on at least one image feature of the image (see 
page 335, column 1). 

As per claim 11, Jagielski teaches of a method wherein generating an aesthetic score 
based on at least one image feature of the image comprises generating an aesthetic score based 
on at least one image feature selected from the group essentially consisting of: color presence, 
color distribution, geometrical quantities of segmented image parts, coefficients of image 
transformations, and higher-level image representations (see page 335, column 1). 

As per claim 12, Jagielski teaches of a method wherein utilizing a classifier comprises 
utilizing one of a Bayesian classifier, a Support Vector Machine (SVM) classifier, a neural net 
classifier, and a decision tree classifier (Abstract; and page 336, column 1). 
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As per claims 13-18, the same limitations are subjected to in claims 1 and 4-8, therefore 
the same rejections apply (see claims 1, and 4-8 above). 

As per claim 19, the same limitations are subjected to in claim 1, therefore the same 
rejections apply (see claim 1 above). 

As per claims 20-21, the same limitations are subjected to in claims 6-7, respectively, 
therefore the same rejections apply (see claims 6-7 above). 

As per claims 22-23, the same limitations are subjected to in claims 9-10, respectively, 
therefore the same rejections apply (see claims 9-10 above). 

As per claim 24, the same limitations are subjected to in claim 12, therefore the same 
rejections apply (see claims 12 above). 

As per claims 25-27, the same limitations are subjected to in claims 1, 4 and 5, 
respectively, therefore the same rejections apply (see claims 1, 4 and 5 above). 

As per claims 28 and 29, the same limitations are subjected to in claims 16 and 18, 
respectively, therefore the same rejections apply (see claims 16 and 18 above). 

As per claim 30, Jagielski teaches of generating aesthetic scores (see claim one above), 
but fails to explicitly disclose generating recommendations based on manipulating visual 
elements in the image. 

However, Yasuda teaches of generating recommendations on how to improve the 
aesthetic score by manipulating visual elements/characteristics in an image (see sections 5-6 on 
pages 540-541, especially section six on page 541). 

It would have been obvious to one of ordinary skill in the art, at the time of the 
applicant's invention, to combine Jagielski's method of generating an aesthetic score for input 
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images based on a trained neural network, with Yasuda's method of generating a 
recommendation based upon manipulating visual elements in an image, in order to provide a 
method for presenting to the user and/or designer, a recommendation as to some of the aesthetic 
options, which when applied to a project, will provide greater aesthetic value. 

5. In response to applicant's remarks that the references fail to show certain features of 
applicant's invention, it is noted that the features upon which applicant relies (i.e., ". . . 
'suggestions as to how the image's score could be improved by manipulating visual elements in 
the images' (specifications, page 14, lines 9-12)"; and "... 'may suggest that particular colors be 
used, or that certain geometrical elements be removed, in order to improve the image's aesthetic 
score' (specification, page 14, lines 12-14)") are not recited in the rejected claims. Although the 
claims are interpreted in light of the specification, limitations from the specification are not read 
into the claims. See In re Van Geuns, 988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

6. In the remarks, Applicants argue in substance that the cited reference fails to discuss a 
method of generating a recommendation to improve the aesthetic score for an image, 

7. In response to the Applicant's argument, the Examiner respectfully disagrees. Jagielski 
teaches of using back-propagation in order to fine-tune neural networks for test cases (see claim 
one above and previous Office Action; page 336, column 2, paragraph four through page 337, 
column one, paragraph one; and Tab 2 on page 337). Regarding generating recommendations to 
improve aesthetic scores, Yasuda clearly teaches of evaluating the end results and compiling a 
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recommendation that can be applied to a design to improve a project (see claim one above and 
previous Office action; and sections 5-6 on pages 540-541, especially section 6 on page 541). 

It would have been obvious to one of ordinary skill in the art, at the time of the 
applicant's invention, to combine Jagielski's method of generating an aesthetic score for input 
images based on a trained neural network, with Yasuda's method of generating a 
recommendation based upon results from the network, in order to provide a method for 
presenting to the user and/or designer, a recommendation as to some of the aesthetic options, 
which when applied to an image/project, will provide greater aesthetic value. Merging the two 
methods also provides the user and/or designer the flexibility of either adding or deleting 
determined options to better refine an image/project. 

Conclusion 

8. The following prior art made of record and not relied upon is considered pertinent to 
applicants disclosure: 

A. Machado et al., "Computing Aesthetics"; 

B. Oh et al., "Image Thresholding by Indicator Kriging"; and 

C. Reich, "A Model of Aesthetic Judgement in Design". 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy 
as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
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MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

10. An inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kelvin Booker whose telephone number is (703) 308-4088. The examiner 
can normally be reached on Monday-Friday from 7:00 AM-5:30 PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Anil Khatri, can be reached on (703) 305-0282. The fax number for the organization 
where this application or proceeding is assigned is (703) 872-9306. 

An inquiry of a general nature or relating to the status of this application proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 



K.E.B. 

Art Unit 2121 
March 15, 2004 




